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What is a Computerized Adaptive Assessment? 


Computerized adaptive testing (CAT), which was first developed four decades ago’, begins with a large pool of 
questions and then selects individual questions for test takers, depending on their responses as they progress. As the 
test taker answers questions correctly, the questions become more difficult. As the test taker answers incorrectly, the 
questions become less difficult. CAT requires the following components: a pool of questions to draw from, calibrated 
to a common measurement scale; a mechanism to select questions on the basis of students’ responses; a process to 


score students’ responses; a process to terminate the test; and reports that relate students’ scores to their instructional 
needs. 


Limitations 


Even with the increased sophistication of today’s CAT programs, there are some limitations to these programs and 
their functions. 


@ The primary limitation to CAT is ensuring that schools have the network infrastructure to successfully 
implement a web-based adaptive testing model. A related issue is providing all students with the required 
devices to take an adaptive test, such as laptops or tablets. 


# Students cannot skip questions or perform a post-test review. Conventional tests allow a student to return to 
a question later after completing other items, while CAT requires the test questions to be answered as they 
appear. 


# Item bank size limits application of CAT. The number of test questions required to cover all the skills taught in 
school over a range of difficulty limits CAT use to entities with substantial resources. Large effort is required to 
maintain a CAT item bank and keep the item content fresh and up-to-date as well as secure. 
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361-375. 


April 2019 


CSAI Brief Implementation of Computerized Adaptive Assessments 


Benefits 


In a computer-based fixed-form test, students will complete a fixed number of test questions. Each student may or may 
not see the same questions, and the item order can be randomized to discourage cheating. These computer-based 
tests share some benefits with CAT, including immediate scoring and paper resource savings. Adaptive testing offers 
additional benefits compared to computer-based fixed-form tests. The following four elements of CAT provide context 
to the discussion of the advantages of CAT over other testing methods. 


@ Item selection that adapts to performance: With CAT, it is possible to use feedback from previous responses to 
select new items. This means that the CAT will become easier if a student is struggling and more difficult if a 
student is performing well. 


4 Improved efficiency: Adaptive testing makes it possible to administer questions near a student's ability level 
and provides more information about student performance compared to other testing methods. This improves 
the precision of measurement and reduces the number of questions that a student must answer. Students 
need less time in front of a computer to complete the test, and teachers have more time for instruction. The 
end result is a test that is both shorter and more accurate than other, more conventional tests. 


@ Test security and item bank maintenance: The matching of items to students results in different item content 
being seen by different students and reduces the risk of cheating in the classroom. The reduced test length for 
CAT also reduces item exposure and risks of students being coached on frequently seen items. 


@ Results delivered immediately: Another advantage of CAT is the immediate calculation of ability level and 
feedback for both students and teachers. Additionally, when CAT is used for formative assessment, a student's 
past performance can be used as a starting point for subsequent exams, to measure growth and to target 
instruction to areas of student need. 


Computerized Adaptive Testing in Practice 
¢ Implementing CAT involves significant education of students, parents, and teachers. 


@ Large investment in technology may be required, to ensure that schools will have enough devices and 
bandwidth for CAT. 


% Significant item development is required. For example, for a 40-question test, an adaptive test bank might 
contain 800 items. 


¢ Existing training will need to be updated for teachers to serve as test administrators. 


@ CAT enables educators to shape teaching strategies, classroom by classroom and student by student, using 
data from the adaptive assessments. These data can be especially effective for instruction when aligned to 
curriculum and state standards. 


@ CAT data, along with information from other assessments, play an integral role in instructional decision- 
making. 


@ School administrators must provide teachers with time to examine student data and to collaboratively build 
responsive instructional strategies. 


@ Data may allow for teachers to group students similarly, or at intentionally mixed-levels, for appropriate 
interventions. 


CSAI Brief Implementation of Computerized Adaptive Assessments 


Teachers are able to look at the strands within CAT to see where students are performing the best and the 
worst comparatively, which can help with instructional design. 


School staff can use CAT data to encourage students to take greater ownership of their learning, and CAT data 
can be part of the shared instructional vocabulary of students, teachers, and parents. 


Policy Questions to Consider 
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Will all schools have the technological infrastructure to support CAT? 


Will the cost of the new assessments exceed current expenditures? If so, how will the state or the district pay 
for these new assessments? 


Will all schools use both CAT formats for summative and formative assessments? 

How will CAT data be used to inform instruction and policymaking? 

How will new teachers, principals, and administrators be trained to use and understand CAT? 

How will experienced teachers, principals, and administrators be trained to use and understand CAT? 


How will the assessments affect compliance with Every Student Succeeds Act accountability provisions? 
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